Schmidt-Hieber J - Nonparametric regression using deep neural networks with ReLU activation function

• The main theorem states that a network estimator is minimax rate optimal (up to log factors) if and only if the method almost minimizes the empirical risk

• Minimax Estimator: δ^M , is an estimator which performs best in the worst possible case allowed in the problem.

\begin{equation} \sup_{\theta \in \Theta} R(\theta,\delta^M) = \inf_\delta \sup_{\theta \in \Theta} R(\theta,\delta). \, \end{equation}

• TODO Wavelet Series Estimators:

• Optimal Estimation Rates for ML NNs with ReLU activation.
• Wavelet estimators can only achieve suboptimal rates under the composition assumption

• Results limited to ReLU activate MultiLayer Feed Forward Neural Networks